A Cross-Linguistic Study on the Production of Multimodal Referring Expressions in Dialogue

This paper presents a cross-linguistic data elicitation study on fully realised referring expressions (REs) in a dialogue context. A webbased experiment was set up in which participants were asked to choose REs to be uttered by one of two agents for identifying five targets in a scripted dialogue. Participants were told that the agent would point at the referents while uttering their chosen linguistic descriptions. The study was conducted in English, Japanese, Portuguese and Dutch and yielded a total of 1190 referring expressions. Our hypotheses concern sets of objects that need to be considered for identification depending on the effect of the pointing gesture. Results show interesting and significant differences between the language groups.